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[57] ABSTRACT 

A method and apparatus for compressing a dissolve 
region within a sequence of digital motion video frames. 
A function is calculated based on statistical data derived 
from frames in the sequence. Dissolve region bound- 
aries are then detected based on characteristics of the 
calculated function. Consecutive detected boundaries 
may be flagged as a dissolve region based on the aver- 
age value of the function between the consecutive de- 
tected boundaries and the distance between the consec- 
utive detected boundaries. 
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, the sequence representing the statistical variance of 

METHOD AND APPARATUS FOR DETECTING pixel values within each frame. Dissolve region bound- 

DISSOLVE REGIONS IN DIGITAL VIDEO aries are then detected by determining where the func- 

SEQUENCES tion is parabolic in shape. According to this embodi- 

5 ment, a dissolve region may be determined to exist 

FIELD OF INVENTION between consecutive dissolve region boundaries when 

This invention relates to video signal processing gen- the distance between consecutive detected boundaries is 

eraSy and particularly to systems for providing a com- less than a predetermined threshold, 

pressed digital video signal representative of a full color In a still further embodiment, intermediate processing 

video signal. 10 may be employed to enhance the function used for 

BACKGROUND dissolve detection. In this embodiment, the function 

may be filtered to remove remnants of hard scene 

It is common practice in video production to use a changes. If the function is the second order difference 

dissolve region to concatenate two scenes. In such a of a function having values corresponding to the statisti- 

regkm, the second scene gradually appears while the 13 variance of pixel values within each frame, all but 

first scene gradually fades out, and by the end of the ^ first spikc „, any cluster of spikes may be filtered and 

dissolve region, the second scene completely replaces rcmoved . Any detected boundaries or regions may also 

the first. Although dissolve regions can be up to several ^ due to frame which may occur 

seconds long, dissolves of i to 1 second are more com- dufm g nrocessinc 

mon in video production. The dissolve process is typi- 20 61 ' ™-^—t- 

cally called cross-fade. BRIEF DESCRIPTION OF THE DRAWINGS 

Attempting to compress a dissolve region via image fl ^ fl illustrating 

compression using a typical motion compensation algo- * * " °7 * V; .. * 

rithrS results in a high entropy predictioVerror image. ?* °P erat,on of fl P referred cmbodunent of * c P rcscnt 

Encoding such an error image requires more bits than 25 m ^ n J, 10n * . J . , 

usual, which may not be affordable for all frames of a mG ; 2 * a diagram of the variance curve for an 

dissolve region without increasing the average bit-rate. exemplary sequence of digital motion video frames. 

An image that is encoded with an insufficient number of FIG 3 is an expanded diagram showing a portion of 

bits may appear blocky when it is decoded. The vfcribil- the variance curve of FIG. 2. 

ity of the blockiness may vary depending on the length 30 FIG. 4 is a preprocessed second-order difference 

of the dissolve region. Such blockiness can be elimi- representation of the variance curve of FIG. 2. 

nated by first detecting dissolves, then encoding the FIG. 5 shows detected dissolve regions for an exem- 

frames in a dissolve region in a proper or different way. plary sequence of digital motion video frames. 

fJjLT ° f F^JS""? 1 *? PrOVide 6 „ DETAILED DESCRIPTION OF THE 

m f>°* ^.apparatus for detecting dissolve regions 35 P^iS^^ODIMENT 
and boundaries within digital video sequences. r ^ ajm^ v^^n^ * 

Further objects and advantages of the invention will Referring now to FIG. 1, there is shown a flow dia- 
become apparent from the description of the invention gram illustrating the operation of a dissolve detector 
which follows. according. to the present invention for detecting a dis- 

SUMMARY OF THE INVENTION 40 "> We region or a dissolve region boundary within a 

sequence of digital motion video frames. The dissolve 

The present invention is directed to a method and detector of FIG. 1 comprises calculating means 100 for 
apparatus for detecting a dissolve region within a se- calculating a function based on statistical data derived 
quence of digital motion video frames. According to the f rom ^ f rames , ^ dissolve region boundary detec- 
present in vention, a function is calculated based on 45 tion means 130 for detecting from said function whether 
statistical data derived from the frames m the sequence. a dtssolve region boundary is present In the cmbodi- 
In a preferred embodiment, the value of a function is ment of no. h c^lve region boundary detection 
calcidated for each frame m the sequence based on the means 130 detennmes a region boundary to 

JS?*? ^Tlf PU l C v vrfl » wl 5 In SS h ffam ! ( m exist when said function exceeds a first predetermined 

The preferred function is the second order difference of 50 . ^ rn\ vr«« c ika ^i^i.finr, *i„ 

. t^cn having valu« co^ponding to*, ^cal E^TSEX? 2< 

variance of pixel values within each frame. Dissolve * , . "T™ 1 .^T,. 6 . 

region boundaries are then detected based on character- ^ undanes " *»* ™ ^ teaman* ^ d * tai J CC 

istfcs of the calculated function. In the preferred em- tetwc f \ consecunve detected boundaries may also 

bodimcnu dissolve region boundaries are detected 35 P™** In the ^bc^inieiit shown, dissolve region 

where the function has an absolute value which exceeds mc f s 180 fo J determining when said average 

a first predetermined threshold. In this embodiment, value cxcccd f « \^tmd predetermined threshold <T2) is 

consecutive detected boundaries are selected and the *° provided. Dtssolve region flagging means 180 may 

average value of the function between consecutive de- a* 80 function to determine if the distance between con- 

tected boundaries is calculated. A dissolve region is 60 secutrve detected boundaries is less than a third prede- 

detected if the average value exceeds a second predeter- terrain ed threshold (T3). The present invention has 

mined threshold and the distance between the subject . been implemented using an Intel model i860 parallel 

boundaries is less than a third predetermined threshold. processor. 

The first predetermined threshold may be made to vary The mathematical basis underlying the operation of 

in accordance with the local characteristics of the func- 65 the preferred embodiment of the present invention may 

tion. be understood by considering a video signal fi%y,t) that 

In an alternate embodiment of the present invention, contains a dissolve region from scene g(x,y,t) to another 

the value of a function is calculated for each frame in scene h(x,y t t). 
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(6) 


5 Xtf no- 


where x t y and t are continuous variables that represent 
the horizontal, vertical and temporal dimensions, re- 
spectively. The parameter a(t) is set equal to 10 


NSa £ M 
n > M 


where ij, and n are the vertical, horizontal and tem- 
poral indices, respectively. T is the frame duration time, 
and is typically equal to 1/30 second. The parameter 
a(n) equals 


and hence varies linearly with the location of the frame 
t within the dissolve region. If g(x, y, t) and b(x, y, t) are 
assumed to be statistically independent with variances 
cri^t) and trt?{t\ respectively, then, the variance tr/(t) 
of f(x, y, t) is given by the following equation (2): 


IS 


(I - o(0JV(0 + *Ht)<rhHi) tiSt&n 


If each scene is assumed Ergodic, then the variance of 
the images in each scene is constant over the entire 3Q 
scene. Hence, Eq. (2) may be reduced to 


trg 1 t < n 

<r h \0 *>t2 


(3> 


n - N 


In the digital domain, the variance curve of Eq. (3) then 
becomes discrete, and can be written as: 


20 


cr,2 * < JV 

trf? n> M 


(7) 


25 


f0 


o / > ii 


(5) 


The second derivative of the discrete variance curve 
(Eq. (7)) may be approximated by a second-order differ- 
ence equation ¥/(nT): 

*/n7) - <r/<nn - 2ff/(7l*» - 1)) + <r/<7T« - 2)X«a) 


35 


From Eq. (3), it is clear that the variance <rf(t) for 
tistStj is a parabola which is concave upward, and 40 
that this parabolic behavior is confined to dissolve re- 
gions. According to the present invention, a dissolve 
region can therefore be detected by detecting its associ- 
ated parabola. 45 

Talcing the firsthand sec ond derivatives of Eq. (3) 
with respect to t gives 


(M- 


- + trt, 2 ) 


(4) 

r< ri 50 


55 


60 


It can be seen that the first and second derivatives of the 
variance curve (Eq. (3» are not defined at the disconti- 
nuity points t=ti and t=t2- 

During digital video compression, the video signal, 65 
f(x, y, t), is digitized temporally as well as spatially to 
form a digital image sequence, f(i, j, nT), which can be 
written as 


-5fiV** + " who ^ 

0 N>M+ 1 


Thus, the second-order difference of the parabola of Eq. 
(7) is a positive constant value with two negative large 
spikes, one at the beginning and the other at the end of 
the parabola, i.e., at n=N+ 1 and n==M+ 1. These two 
negative spikes are due to the discontinuities at n=N 
and n=M, and may be used in detecting the parabola, 
and hence, the dissolve region. 

Referring again to FIG. (1), there is shown a flow 
diagram illustrating operation of a preferred dissolve 
region detector according to the present invention. 
Calculating means 100, for calculating a function based 
on statistical data derived from a frame sequence, begins 
by calculating the second-order difference of the vari- 
ance curve, ¥/nT) for the subject frame sequence. 
Dissolve region boundary detection means 130 for de- 
tecting from said calculated function whether a dissolve 
region boundary is present, then records negative spikes 
in ¥j(nT) whose magnitudes exceed a first predeter- 
mined threshold Tl. Cluster filtering means 140 for 
filtering said recorded spikes, then removes all but the 
first in every cluster of negative spikes. Means 150 for 
calculating the average value of the calculated function 
between consecutive detected boundaries, next calcu- 
lates the average value of Y/nT) within the regions 
defined by each pair, of consecutive negative spikes. 
Means 170 for determining the distance between con- 
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secutive detected boundaries, next computes the width was approximated by the second order difference 
of the regions defined by each pair of consecutive nega- V/nT) given in Eq. (8). The value of the thresholds T2, 
tive spikes. Dissolve region flagging means 180 for T3, and T4, and of L and kj were empirically chosen 
determining whether said average value exceeds a sec- and set to be 3, 2, 45, 21 and 3.5, respectively. FIG. 2 
ond predetermined threshold, compares the calculated 5 shows the variance curve for the "Comp B" sequence. 
. average value for a subject pair of consecutive detected FIG. 3 is a blowup of the "Comp B" sequence variance 
spikes with a second predetermined threshold T2. In the curve showing the dissolve region between frames 466 
preferred embodiment, dissolve region flagging means and 480. The preprocessed second derivative of the 
180 also compares the calculated distance between a exemplary variance curve is shown in FIG. 4 where the 
subject pair of consecutive detected spikes with a third to negative spikes around each dissolve region can be seen, 
predetermined threshold T3. In the embodiment shown, During preprocessing of the second-order difference 
if the calculated average value exceeds T2, and if the curve, the spikes at frames 86, 552, and 701 were re- 
calculated distance between the two subject spikes is moved as they were due to hard scene changes; the 
less than T3, a dissolve region is flagged for all the spikes at frames 17,355, and 368 were also removed. In 
frames between and including the two spikes. T3 is used 15 FIG. 5, the detected dissolve region boundaries for the 
to eliminate long dissolves which need not be flagged exemplary sequence are shown, 
for special processing. In the preferred embodiment described above, calcu- 

In the preferred embodiment of the present invention, la ting means 100, for calculating a function based on 
the value of Tl is adapted to the local characteristics of statistical data derived from a frame sequence, began by 
V/nT). For frame n, Tl— — krfor^+m^, where ly is a 20 calculating the second-order difference of the variance 
positive constant ov and mj are the standard deviation curve for a frame sequence to be encoded. In alternate 
and the mean, respectively, of */nT) either in a win- embodiments, the function calculated by the calculating 
dow of length L, that is centered around, but excluding means may be a representation of the first-order differ- 
fxame n, or in a window of length L/2 that is adjacent ence of the variance curve of a frame sequence, or may 
to frame n. The mean and standard deviation of the 25 simply be a representation of the variance curve cone- 
window centered around frame n are used if the differ- spending to a frame sequence. Other higher-order dif- 
ence between the means of the left and right adjacent ferential representations of the variance curve may also 
windows is less than a fourth predetermined threshold, be employed. In addition, although the derived statisri- 
T4. However, if it is not, then the mean and the standard cal data used in the preferred embodiment came di- 
deviation of the adjacent window with the smallest 30 recti y from a frame sequence being encoded, in alter- 
mean are used. Blocks 131-134 illustrate this preferred nate embodiments, such derived statistical data could be 
method for calculating o*d and mjon a frame by frame derived from displacement vectors, difference images, 
basis. It is to be understood that in alternate embodi- error images or the like which themselves are derived ' 
merits, Tl need not be made to vary in accordance with from a frame sequence being encoded. Such statistical 
the local characteristics of the function for determining 33 data could also be derived from the low frequency or 
whether dissolve region boundaries are present. In such high frequency content of a frame sequence being en- 
alternate embodiments, Tl can simply be set at a prede- coded, or from images derived from such high or low 
tennined constant for deterrnining in all instances frequency content. Finally, the derived statistical data is 
whether dissolve region boundaries are present not limited to data representative of variance informa- 

To discriminate between dissolve regions which need 40 tion, but may include standard deviation or other statis- 
not be identified for special handling, some pre-process- . tical data derived from a frame sequence, 
ing of V/nT) is also employed in the preferred embodi- In the preferred embodiment of the present invention, 
ment of the present invention. Thus, bard scene filtering the dissolve region boundary detecting means 130, for 
means 120 may be used to detect and eliminate all nega- detecting from a calculated function whether a dissolve 
tive and positive spikes which are due to hard scene 45 region boundary is present, determines a dissolve region 
changes. In addition, cluster filtering means 140 may be boundary to be present where the calculated function 
used for setting the value of V/nT) at a frame immedi- exceeds a predetermined threshold. In alternative em- 
ately next to a frame that has a negative spike to zero. bodiments, a dissolve region boundary may be detected 

Finally, post-processing of the flagged dissolve re- where the actual (as opposed to absolute) value of the 
gions may also be used to process detected dissolve SO calculated function exceeds a positive threshold, or 
regions to further optimize encoding. In this regard, where the actual value of the function falls below a 
two adjacent dissolve regions that can form one contin- negative threshold. Where the function calculated by 
sous region may be identified and the region with the the calculating means is simply a representation of a 
smaller average second derivative value between its variance curve derived from a frame sequence, a dis- 
spikes may be ignored. Two regions can not generally 55 solve region boundary may be detected by determining 
be merged together to form one dissolve, because a where that function is parabolic in shape. Similarly, if a 
negative spike is not expected within a dissolve region, first-order difference representation of a variance curve 
since the value of ¥/nT) for the intermediate frames is is used as the calculated function, dissolve region 
typically a positive constant Also, an adjustment stage boundaries may be determined to be present where the 
may be employed to compensate for a one-frame shift in £0 calculated function is discontinuous, 
detected dissolve regions due to approximating the In the preferred embodiment, if dissolve region 
second derivative with a second-order difference opera- boundary detection means 130 detennines a pair of 
tion. consecutive boundaries to exist, further analysis is pcr- 

A dissolve detector according to the present inven- formed in the region between that pair of detected 
tion was implemented in connection with a test video 65 boundaries before determining whether to flag the re- 
image sequence referred to as the "Comp B" sequence. gion between that pair of detected boundaries as a dis- 
The variance of each image was first computed to form solve region. In particular, in that embodiment the cal- 
the variance curve. The second derivative of this curve culated average value between the pair of detected 
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boundaries was required to exceed T2 and the distance 9. The method of claim 1, wherein said dissolve mea- 

between the pair of detected boundaries was required to sure is calculated in step (b) based on the statistical 

be less than T3 before a dissolve region was flagged. In variance of pixel values within each of said selected 

alternative embodiments, a dissolve region may be frames. 

flagged or detected simply by detecting its associated 5 10. The method of claim 9, wherein said dissolve 

boundaries. In such embodiments, a dissolve region may . measure is the second order, difference of the statistical 

be detected or flagged solely based on where a calcu- variance of pixel values within each of said selected 

lated function exceeds or falls below a predetermined frames. 

threshold, where a calculatrd function is discontinuous, n. The method of claim 10, wherein said first prede- 

or where a calculated function is parabolic in shape. In 10 tennined threshold varies in accordance with the local 

further alternative embodiments, the width of a de- characteristics of said dissolve measure and step (c) 

tected or flagged dissolve region may span between and further comprises determining said first predetermined 

including detected boundary pairs, may span between threshold for at least one point of said dissolve measure, 

but excluding such pairs, or may span somewhat be- 12. The method of claim 11, wherein step (c) further 

yond such pairs. As noted above, the width may be 13 comprises the steps of: 

frame-shifted due to differential or other approximating i calculating a first mean representing the mean of 

or processing operations. said dissolve measure within a first window adja- 

Thc present invention may be embodied in other to ^ at least one point; 

specific forms without departing from the spirit or es- y calculating a second mean representing the mean of 

sential attributes of the invention. Accordingly, refer- 20 ^ d i SiQ \ vc measure within a second window 

ence should be made to the appended claims, rather adjacent to said at least one point; 

than the foregoing specification, as indicating the scope ^ calculating the difference of the result of step 0) 

of the invention. and the result of step (ii); and 

What is claimed is: . . « iv. comparing said difference to a fourth predeter- 

1. A method for compressmg a dissolve region within 25 : mined 'thr«hoId and if said difference exceeds said 
s sequence of digital motion video frames compnsmg fourth predetermined threshold then 

me steps of: determining said first predetermined threshold 

(a) selecting a plurahty of frames from said sequence from ^ m ean and standard deviation of said 

(b) caSa^fdissolve measure based on statistical 30 dissolve measure within a window centered 
^daTdenVed from said selected frames; a*ut said at least one pomt but excluding said at 

(c) detecting dissolve region boundaries within said least on point 

sequence by detennining where the absolute value otherwise , 

of said dissolve measure exceeds a first predeter- determining said first predeternnned threshold 

mined threshold; and 35 from , ^ mcan standard deviation of said 

(d) compressing at least one of said selected frames in ^ Wc * e ** dow 
accordance with the result of step (c). corresponding to the smaller of said first and 

2. The method of claim 1, wherein step (c) further if 001 " 1 P 10 ? 03 : , . K ' . - . 
comprises the step of: 13 The method of claim 6, wherein step (b) further 

selecting at least one pair of consecutive detected 40 comprises the steps of: 

boundaries and for said at least one pair of consecu- i. detecting all spikes m said dissolve measure which 

tive detected boundaries due to ***** scene changes; and 

i. calculating the average value of said dissolve **■ removing fronvsaid dissolve measure all spikes 

measure between said consecutive detected detected in step 0). . . 

boundaries; and 45 **• TT* e method of claim 10, wherein said detected 

ii flagging a dissolve region when , said average boundaries have been frame shifted and step (c) further 

value exceeds a second predetermined threshold. comprises adjusting said detected boundaries to com- 

3. The method of claim 2, wherein a dissolve region pensate for said fraine shifting. \- 

is flagged in step (cXii) only when the distance between 15 The method of claim 1, wherein said statistical 

said consecutive detected boundaries is less than a third 50 data is derived from motion compensation displacement 

predetermined threshold. vectors generated from said selected frames. 

4. The method of claim 3, wherein the width of said The method of claim 1, wherein said statistical 
detected dissolve region in step (cXii) spans between data is derived from difference images generated from 
and including said detected consecutive boundaries. Mid selected frames. 

5. The method of claim 3, wherein the width of said 55 17. The method of claim 1, wherein said statistical 
dissolve region in step (cKu) is greater than the width data is derived from motion compensation error images 
spanning between said detected consecutive bound- generated from said selected frames. 

aries. 18. The method of claim 1, wherein said statistical 

6. The method of 4, wherein dissolve region data is derived from the high frequency content of said 
boundaries are detected in step (c) at spikes where the 60 selected frames. 

absolute value of said dissolve measure exceeds said first 19. The method of claim 1, wherein said statistical 

predetermined threshold. data is derived from the low frequency content of said 

7. The method of claim 6, wherein step (b) further selected frames. 

comprises removing from said dissolve measure all but ■ 20. A. method for. compressmg a dissolve region 

the first spike in any cluster of spikes. 65 within a sequence of digital motion video frames com- 

8. The method of claim 7, wherein step (b) further prising the steps of: 

comprises setting the value of said dissolve measure to (a) selecting a plurality of frames from said sequence 

zero at each point immediately next to a negative spike. for processing; 
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(b) calculating a dissolve measure based on statistical 
data derived from said selected frames; 

(c) detecting dissolve, region boundaries within said 
sequence by determining where said dissolve mea- 
sure first decreases then increases in a parabolic 
fashion; and 

(d) compressing at least one of said selected frames in 
accordance with the result of step (c). 

21. The method of claim 20, wherein step (c) further 
comprises the step of: 

selecting at least one pair of consecutive detected 
boundaries and for said at least one pair of consecu- 
tive detected boundaries 

L determining the distance between said consecu- IS 

tive detected boundaries; and 
ii. flagging a dissolve region when said distance 
between said consecutive detected boundaries is 
less than a predetermined threshold. 


10 


10 


30. The method of claim 20, wherein said statistical 
data is derived from the low frequency content of said 
selected frames. 

31. An apparatus for compressing a dissolve region 
within a sequence of digital motion video frames com- 
prising: 

(a) means for selecting a plurality of frames from said 
sequence for processing; 

(b) means for calculating a dissolve measure based on 
statistical data derived from said selected frames; 

(c) dissolve region boundary detection means for 
detecting the presence of dissolve region bound- 
aries within said sequence from said dissolve mea- 
sure; and 

(d) means, responsive to said dissolve measure, for 
compressing at least one of said selected frames. 

32. The apparatus of claim 31, wherein said dissolve 
region boundary detection means detects a dissolve 
region boundary where said dissolve measure has an 


22. The method of claim 21, wherein the width of said 20 absolute value which exceeds a first predetermined 


dissolve region in step (c)(ii) is equal to the width be- 
tween and including said detected consecutive bound- 
aries. 

23. The method of claim 21, wherein the width of said 
dissolve region in step (c)(ii) is greater than the width 
spanning between and including said detected consecu- 
tive boundaries. 

24. The method of claim 20, wherein step (b) further 
comprises the step of filtering said dissolve measure. 

25. The method of claim 20, wherein said dissolve 
measure is the statistical variance of pixel values with 
each of said selected frames. 

26. The method of claim 20, wherein said statistical 


25 


30 


threshold. 

33. The apparatus of claim 32, wherein said dissolve 
region boundary detection means further comprises: 

(i.) means for selecting at least one pair of consecutive 
detected boundaries; 

(ii.) means for calculating the average value of said 
dissolve measure between said at least one pair of 
consecutive detected boundaries; and 

(iii.) dissolve region flagging means for determining 
when said average value exceeds a second prede- 
termined threshold. 

34. The apparatus of claim 31, wherein said dissolve 
region boundary detection means detects a dissolve 
region boundary where said dissolve measure first in- 


data is derived from motion compensation displacement 35 creases then decreases in a parabolic fashion. 


40 


vectors generated from said selected frames. 

27. The method of claim 20, wherein said statistica] 
data is derived from difference images generated from 
said selected frames. 

28. The method of claim 20, wherein said statistical 
data is derived from motion compensation error images 
generated from said selected frames. 

29. The method of claim 20, wherein said statistical 
data is derived from the high frequency content of said 45 
selected frames. 


35. The apparatus of claim 34, wherein said dissolve 
region boundary detection means further comprises: 
(i.) means for selecting at least one pair of consecutive 

detected boundaries; 
(ii.) means for calculating eh average value of said 
dissolve measure between said at least one pair of 
consecutive detected boundaries; and 
(iii.) dissolve region flagging means for determining 
when said average value exceeds a second prede- 
termined threshold. 
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UNITED STATES PATENT AND TRADEMARK OFFICE 

CERTIFICATE OF CORRECTION 

PATENT NO- : 5,283,645 
DATED : February 1, 1994 

INVENTOR(S) : Admn M. Alattar 

It is certified that error appears in the above-indentified patent and that said Letters Patent is hereby 
corrected as shown below: 

Column 3, line 19, -o&(tf should read -og 2 ^)-- 

Column 3, line 37, "oh 2 ®" should read -o\-. 

Column 4, line 30 "(8a) w should identify the equation, not be pan of it 

Column 7, line 26 *V should read -a-. 

Column 10, line 40 "eh" should read -the-. 
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